FIG. 1 is a block diagram of a McCollough-Pitts (MP) type neuron widely in use today. The unit has a set of parallel input ports 11 for accepting an N-element input vector, [u.sub.1, u.sub.2, . . . , u.sub.N ].sup.T. Weighing elements 13 scale (or multiply) their corresponding input vector elements to form a set of product terms, {u.sub.j w.sub.ij }, each of which is applied to adder unit 15 to form the vector dot-product, [u.multidot.w]=.SIGMA.u.sub.j w.sub.ij. Also, an offset or bias term, .theta..sub.i, applied to input 12 may be added to form the sum [u.multidot.w].sub.i +.theta..sub.i. Each input branch consisting of an input data element 11 and a weighting element 13 corresponds to a neural synapse. The sum out of adder 15 is applied to a nonlinear element 19, usually having a sigmoidal transfer characteristic to form the neuron cell output signal.
One realization of the synapse portion of the MP type neuron shown in FIG. 1 may be found in U.S. Pat. No. 4,956,564, Sep. 11, 1990, by Holler et al. The patent discloses "a synapse cell for providing a weighted connection between an input voltage line and an output summing line having an associated capacitance". In this case, the "associated capacitance" performs the summing function of adder 15. A single synaptic element of the above reference, shown in FIG. 2, is a four quadrant multiplier using floating gate transistors 40-43, differential input lines 50 and 51, and differential output lines 44 and 45. The drains of transistors 40 and 42 are coupled to current summing line 52 while the drains of transistors 41 and 43 are coupled to current summing line 53. Summing line 52 and 53 have at nodes 44 and 45 associated capacitances 46 and 47, respectively, for accumulating charge. Select devices 60 and 62 are n-channel transistors for selectively connecting the sources of floating gate transistors 40 and 42 to line 64; devices 61 and 62 selectively connect transistors 41 and 43 to line 65. Return lines 64 and 65 are also used for erasing and normal reading operations.
For positive inputs, a voltage pulse of t.sub.pp seconds is applied to input line 50 while for negative inputs a voltage pulse of t.sub.pn seconds is applied to input line 51. The duration of the pulse is made proportional to the magnitude of the input signal. The charge associated with the current flow through the floating-gate devices is representative of the product of the input pulse duration and the stored weight in the floating gate devices. The stored charge on capacitances 46 and 47, and hence voltage across 46 and 47, represent the four quadrant multiplier output values in differential form, i.e., the voltage difference between output lines 44 and 45.
For each input synapse of a given neuron, a structure similar to that shown in FIG. 2 is required. FIG. 3 shows a typical single layer neural network with N input channels and M neurons. Thus, N.times.M multiply units 13 are required, each similar to that shown in FIG. 2 or its functional equivalent. Line 21, the input line to accumulator 15, would be a differential input line pair, corresponding to lines 44 and 45 of FIG. 2, if that type of differential multiplier were to be used. Similarly, input line 11 would correspond to differential input lines 50 and 51.
Other forms of multiplier (weighting) elements may be used, based on the four quadrant transconductance multiplier cell shown in FIG. 4. (See J. Watson, "Analog and Switching Circuit Design", Adam Hilger Ltd, Bristol, 1984, pp. 134-140). The product of inputs 102 and 77 are represented by the current difference, .DELTA.I, which corresponds to the difference in collector currents flowing in leads 71 and 73. Thus, assuming matched transistors (81-86), ##EQU1## where q is the charge on an electron,
k is Boltzman's constant, PA1 T is the absolute temperature.
For x small, tanh x.apprxeq.x so that equation (1) may be approximately represented by ##EQU2##
One synapse implementation shown in FIG. 5, the commercially available Intel Model 80170NX electrically trainable analog neural network (manufactured by Intel Corporation, 3065 Bowers Avenue, Santa Clara, Calif., 95051), is based on a transconductance four quadrant multiplier using CMOS technology. Transistors 93 through 98 form a four quadrant multiplier unit similar to the type described above. The differential input vector component, ui, is applied by input lines 102 and the differential weights, w.sub.ij.sup.+ and w.sub.ij.sup.-, supplied by EEPROM cells 900, are applied to the gates of transistor 93 and 94, respectively. Select line 92 activates cells 900 by applying Vcc to the gates of transistors 91 and 99 that act as control switches for applying the drain voltage, Vd, and the control gate voltage, Vcg, to floating gate transistor 90 of each EEPROM cell 900. By selecting appropriate values of Vcg, Vd and source voltage, Vs, stored weights values (w.sub.ij.sup.+, w.sub.ij.sup.-) in the floating gate transistors 90 may be modified or applied to transistors 93 and 94 of the four quadrant multiplier. The differential output current, .DELTA.I.sub.j =I.sub.j.sup.+ -I.sub.j.sup.-, available on output leads 21, is representative of the product u.sub.j * w.sub.ij, where w.sub.ij =w.sub.ij.sup.+ -w.sub.ij.sup.-.
As in the previously described synapse implementation by Holler et al., a single layer network of N inputs and M neurons requires an interconnected matrix of N.times.M synapse cells.
To change the weights stored in the differential pair of EEPROM cells 900, charge is added to one cell's floating gate and subtracted from the other using Fowler-Nordheim tunneling. Tunneling is commonly used in conventional EEPROM devices by applying high voltages across a thin dielectric causing electrons to "tunnel" directly through the dielectric. The Intel model 80170NX synapse multiply characteristics, shown in FIG. 6, are plot of differential output current, .DELTA.I.sub.j, versus input voltage, u.sub.j, for various values of differential voltage stored in the floating gates of the EEPROM cell pair.
Because artificial neurons find application to problems requiring substantial parallel processing (many neural cells simultaneously active), the above implementations have the disadvantage of requiring N.times.M multipliers for each neural network layer of N input channels (input vector elements) and M neurons. This requirements limits the capacity (N.times.M) of any integrated circuit implementation. Also, accommodation of large input vectors and output vectors is difficult because of the large number of pins required. These two problems provide motivation for the present invention to be described.